2 links tagged with all of: local inference + decision models
Click any tag below to further narrow down your results
Links
Strands released Strands Decider 2B, a 2-billion-parameter model designed to quickly pick between predefined options rather than generate arbitrary text. It runs locally in ~115ms and ranks third among 2B-class decision models on accuracy and calibration benchmarks.
- Decision models sacrifice flexibility for speed and reliability: they choose from given options and assign confidence scores, making them 100x faster than LLMs for simple classification tasks but unsuitable for text generation, coding, or reasoning.
- The 2B parameter size hits a sweet spot for local experimentation—small enough to run on consumer hardware, large enough to handle meaningful work like tool selection, routing, and guardrails in agentic workflows.
- Strands open-sourced the full model, training data, and 19 iterations of architecture refinements on GitHub and Hugging Face, explicitly inviting community contributions to the decision model space.
Kev is a family of small language models (0.8B to 9B parameters) built on Qwen3.5 that make yes/no, multiple-choice, and rating decisions on text inputs. You can run them locally on your own hardware with an API compatible with TypeSafe's System One, plus there's a web playground for testing.
- Three model sizes available with published accuracy metrics: Kev-9B reaches 85.2% accuracy on unseen datasets and trails the proprietary Jev model by only 3.5 points, while Kev-4B and Kev-0.8B offer smaller footprints for different use cases.
- All models run on consumer hardware (CUDA, ROCm, Apple Silicon) with the 4B and 9B versions fitting on a 32GB Mac in bf16 precision, plus you can train your own weights using provided code and evaluation data.
- Probabilities are calibrated by default and the models output confidence scores alongside answers, reducing confident errors to 4% on new data compared to 8.7% without calibration.